Upcoming Event: Oden Institute Seminar

Curvature-Aware Optimization for High-Accuracy Physics-Informed Neural Networks

Khemraj Shukla, Research Associate Professor, Brown University

Abstract

Efficient and robust optimization is essential for training neural networks, enabling scientific machine learning models to converge rapidly to high accuracy while faithfully capturing complex physical behavior governed by partial differential equations (PDEs). In this work, we present advanced optimization strategies to accelerate the convergence of physics-informed neural networks (PINNs) for challenging PDEs.  

Specifically, we develop efficient implementations of the Natural Gradient optimizer, as well as self-scaling BFGS and Broyden quasi-Newton methods, and demonstrate their performance on benchmark problems including the Helmholtz equation, Stokes flow, the inviscid Burgers equation, and the Euler equations for high-speed flows. Beyond optimizer development, we also propose new PINN-based formulations for solving the inviscid Burgers and Euler equations, and compare the resulting solutions against high-order numerical methods to provide a rigorous and fair assessment.  

Finally, we address the challenge of scaling these quasi-Newton methods for batched training, enabling efficient and scalable solutions for large-scale scientific machine learning problems.

Biography

Raj received his Ph.D. in computational geophysics from Oklahoma State University (OSU), where he studied high-order numerical methods for poroelastic systems in collaboration with Prof. Jesse Chan at Rice University. He is currently a Research Associate Professor in the Division of Applied Mathematics at Brown University (Providence, RI, USA). His research focuses on scientific machine learning (SciML) for shape optimization of hypersonic vehicles, high-order numerical methods, low-Mach-number reactive flows, and turbulence modeling.